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The three articles in this newsletter focus on 
questions of what teacher knowledge consists of and how it can be 
measured. Three contemporary efforts to assess teacher knowledge, 
each with a different purpose, are presented. The first, based on a 
report of the Office of Educational Research and Improvement, 
summarizes state assessment efforts and issues raised by such 
efforts. The second article describes Michigan State University's 
National Center for Research on Teacher Education's approach to 
instrument development, detailing how the organization conceptualized 
what teachers need to care about, know, and be able to do to teach 
mathematics and writing effectively, and how the development o£ 
instriunents to assess teachers' learning is approached. The question 
of what it means to be prepared to teach an academic subject is also 
addressed. The final article offers an interview with Lee Shulman of 
Stanford University (California) who describes his ongoing work with 
the Carnegie Forum on Education and the Economy, which is developing 
assessment prototypes for a national board examination of teachers. 
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The National Center for Research on Teacher Education is 
committed to research that will contribute to the improve- 
ment of teacher education. Headquarters are at Michigan 
State University, an institution with a history of 
innovative programming in teacher education. 

In pursuit of its mission, the NCRTE cooperates with teach 
educators engaged in a variety of approaches and alterna- 
tives to teacher education. The programs they represent 
are listed below. 

Participating Sites 

Preser *ice 

Dartmouth College 
Teacher Certification Program 
Hanover, NH 

Illinois State University 
Elementary Education and Secondary Program for Mathematics 
Normal, IL 

Michigan State University 
Academic Learning Program 
East Lansing, MI 

Norfolk State University 
Early Childhood and Elementary Education 
Norfolk, VA 

Trenton State College 
Elementary Education 
Trenton, NJ 

University of Florida 

Elementary PROTEACH and Secondary English PROTEACH 
Gainesville, FL 

First Year 

Albuquerque Public Schools/University of New Mexico 
Graduate Intern/Teacher Induction Program 
Albuquerque, NM 

Los Angeles Unified School District 
Teacher Trainee Program 
Lof Angeles, CA 

New Jersey Provisional Teacher Program 

Inserfice 

Columbia University 
Teachers College Writing Project 
New York, NY 

Mount Holyoke College 
SummerMath for Teachers Program 
South Hadley, MA 
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The National Center for Research on Teacher Education, funded 
by a grant fr -n the Office of Educational Research and Im- 
provement, Department of Education, is committed to examining 
the relationship between teacher education and teacher 
learning. 

The Center's research agenda can be characterized by three 
critical features. First, it contrasts tv/o academic subjects 
(mathematics and writing) in order to increase our understand- 
ing of the relationship between subject m'^tter and pedagogy in 
learning to teach. Second, it includes a variety of approaches 
and alternatives to teacher education in the sample* so that 
we can see the different ways in which teaching may be 
learned. Third, it documents changes in teachers* knowledge, 
skills, a&d dispositions as they participate in these programs, 
so that we can see the relationship between educational 
experiences and changes in knowledge. 

Study Design The Center's work consists of case studies of a variety of 

approaches and alternatives to teacher education combined with 
longitudinal studies of teachers as they participate in these 
programs and enter their teaching. We include undergraduate 
and gradaate-level preservice programs, alternative routes to 
teaching, and inservice programs. These approaches differ in 
the relative emphasis they place on liberal arts studies, for- 
mal study of pedagogy, and guided practice. They also differ 
in the way they treat academic subject matter and pedagogy. 

The longitudinal studies will enable us to examine changes 
within individuals over time (e.g., in knowledge of mathema- 
tics and writing and in knowledge about how to teach these 
subjects) and to see whether or how these changes are related 
to particular kinds of learning opportunities, sucn as prac- 
ticum experiences, methods courses, or on-the-job experiences. 

Colloquy, the NCRTE's newsletter, is devoted to examining 
important issues in teacher education. Each issue will pro- 
vide different perspectives and points of view on a particular 
topic. Articles will present research findings as well as 
debates. The first issue focuses on questions of what teacher 
knowledge consists of and how it can be measured. We high- 
light three contemporary efforts to assess teacher knowledge, 
each with a different purpose. 

Forty-four states now require some form of teacher certifica- 
tion exam. One article, based on a report of the Office of 
Educational Research and Improvement, U. S. Department of 
Education, summarizes state assessment efforts and issues 
raised by such efforts. 

The NCRTE is about to launch its study on new efforts to 
assess teacher knowledge. Another article, by Deborih Ball 
and Bill McDiarmid, describes NCRTE*s approach to instrument 
development, detailing how we conceptualized what teachers 
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need to care about, know, and be able to do to teach mathe- 
matics and writing effectively and how we approached the 
development of instruments to assess teachers' learning. The 
authors address the question of what it means ^o be prepared 
to teach an academic subject. 

The National Board for Professional Teaching Standards was 
established on May ^S. It will be issuing certificates to 
teachers who demonstrate they meet standards that can be 
measured. Our third article, an interview with Lee Shulman 
of Stanford University, describes Shulman's ongoing work with 
the Carnegie Forum on Education and the Economy, which 
established the board. Shulmr.i* is developing assessment 
prototypes for a national board examination for teachers. 



\fary Kennedy 



Prior to becoming director of 
the NCRTE. Mary Kennedy 
directed a number of large- 
scale studies, including the 
congressionally-mandated Na- 
tional Assessment of Chapter 1, 
completed in preparation for 
the 1987-88 reauthorization of 
Chapter 1. Throughout her 
career, Kennedy has taken 
particular interest in research 
methodology and in the rela- 
ship between research and 
practice. 
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Responding to reform efforts tt the national level (e.g., 
Carnegie Forum on Education and the Economy. National 
Commission for Excellence in Teacher Education, the Holmes 
Group Consortium), 48 states have implemented or plan to 
implement some form of teacher certification testing. Through 
these efforts, states hope to restore public confidence in the 
educational system and to keep unqualified people out of thr 
profession. In the late 60s and 70s, concern about the qual- 
ity of education led to student testing on a large scale. 
Since then, media coverage and the widespread perception of 
the public that schools were not educating students led to 
concerns about teacher competence. 

A new report from the Office of Educational Research and 
Improvement of the U. S. Department of Education surveys the 
current status of state assessment efforti (Lawrence Rudner, 
Ed^ WhaVs Happening in Teacher Testing: An Analysis of 
State Testing Practices^ 1987). The report points out that, 
by 1987, some 44 states had initiated written certification 
test requirement, 39 had considered or initiated an induction 
program with an assessment component, and 27 had initiated an 
admissions test for teacher education. The report provides a 
state-by-state review of admissions and certification testing, 
as well as a series of papers reviewing social, legal, and 
technical issues of teacher assessment This article summa- 
rizes some of the issues examined in the report. 

In a paper on "Historical Perspectives," J. T. Sandefur of 
Western Kentucky University traces the background of the 
return to teacher testing. In the late nineteenth century, he 
points out, states and counties administered tests for licen- 
sure to assure that high school graduates were qualified to 
teach. State testing fa^ed away as educational requirements 
for teachers were increased, leading finally to an under- 
graduate degree comprised of general education, academic 
specialties, and pedagogical instruction. The current return 
to teacher testing began in the south with seven states and 
has spread to the west and northwest. Initially teacher 
testing programs were mandated by state legislatures, but in 
the last five years they have been mandated by state boards 
of education. 



Rudner, the editor of the volume, points out that many 
stakeholders in education— the public, teachers educators, 
teachers, and their professional organizations-support the 
concept of teacher testing, although they disagree on who 
should be evaluated, what should be evaluated, and how 
evaluation is conducted. 
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What are the tests measuring? How are these tests developed? 
Are performance assessments a valid alternative? These are 
the questions that are addressed by the researchers contribut- 
ing to Rudner's volume. 

In a paper entitled "Content and Difficulty of a Teacher 
Certification Examination," Rudner details the skills assessed 
by tests, how the tests are evaluated, and how passing scores 
are set. The NTE, formerly referred to as the Kational 
Teachers Examination, under the aegis of the Educational 
Testing Service, has served as a model for other paper-and- 
pencil tests and consists of a core battery and exams covering 
specific subjects. The core battery of thr. NTE and most 
certification tests measure general knowledge, professional 
knowledge, and communication skills* General knowledge 
includes reading, writing, mathematics, literature, and fine 
arts. Communication skills deal with listening and receiving 
feedback. Professional knowledge addresses classroom manage- 
ment, student assessment, and legai matters. Tie NTE, as do 
other certification tests, provides specific tests in subject 
matter areas. 

Rudner points out that such tests do not address a number of 
dispositions that may be important to teaching, such as dedi- 
cation, caring, and perseverence. Nor do paper-and-pen^l 
tests measure actual teaching ability. 

Test developers may rely on either content-related and/or 
criterion-referenced evidence to establish the validity of a 
test. 

Content-related evidence refers to evidence that the material 
is related "<o the job of teaching. In a technical appendix, 
James Algina and Sue Legg, University of Florida, point out 
the importance of being clear about what the job is-is it 
teaching in general, acceptable practice for entry-level 
teachers, or effective teaching? Even if one goes through the 
test validation procedure of examining the tasks and the cur- 
riculum, should job-relatedness be the main basis for develop- 
ing a test of basic skills? Parents, for example, may want 
teachers to have higher skills in mathematics and writing than 
those required for "adequate" job performance. 

In a paper entitled "Legal Considerations," Martha McCarthy, 
Indiana University, reviews the legal challenges to testing 
and the principles for assessment that courts have estab- 
lished. Courts have determined that tests can be used for 
teacher employment decisions if the content is job-related and 
if the test is constructed and validated by professionals. 
Test content is usually validated by panels of teachers, 
teacher educators, and administrators appointed by state 
boards of education. Rudner, in his paper on test content, 
says these panels consider whether there is a match between 
the test and what prospective teachers learn in a standard 
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teacher education program. They consider whether skills 
covered by the test are relevant to the job, but not whether 
all crucial job elements are reflected in the test. 

Criterion-related evidence refers to evidence that successful 
test performance i$ related to successful teaching. Certi- 
fication practices often rest on the assumption that higher 
test scores are related to better teaching practice and that 
teachers certified by tests will have stronger positive 
effects on their students. 

Although such criterion-related evidence might yield tests 
with predictive value, critics state that it is difficult to 
obtain valid criterion data because teachers work in a wide 
variety of situations and that measures of teacher performance 
have not yet been well developed. Algina and Legg are 
optimistic about the possibility of finding common criteria; 
low-infeivacc measures are already being developed that may 
prove useful. 

How Arc Standards Set? Standard setting is the process of determining the minimum 

score needed for certification. Typically, a panel of experts 
(!etermine the test performance that should be expected from a 
person of minimal competence. Although these practices give 
the appearance of establishing an objective cut-off score, 
they actually derive from professional judgments, rather than 
from evidence that such scores are dissociated with particular 
levels of teaching skill. Rudner's paper on test content and 
difficulty notes that some states set the cut-off score to 
guarantee a supply of teachers, while in others it has become 
necessary to issue emergency certificates to handle shortages 
created by the test In most cases, states set a lower cut- 
off score than the panel of experts recommends. 

Are Performance Assess- Besides paper-and-pencil tests of teacher knowledge, seven 

ments a Valid Alternative? states have developed performance assessment tests for the 

beginning teacher. In a paper on "Assessment of the Teaching 
Skills of Beginning Teachers,* Gary Galuzzo of Western 
Kentucky University reports that two methods are usually 
employed for developing classroom observation instruments. In 
the "consensus* method, a cross-section of community, profes- 
sional, and some lay people generate a list of competencies 
and then agree on those that should be valuable for a beginner 
to attain. They assume that these can be observed in the 
classroom. The "research review* method has a group of edu- 
cators reviewing the literature to come up with a list of 
research-informed competencies to indicate effective teaching. 
These competencies have typically already been observed with 
instruments developed for research purposes. 

While performance assessments are not identical, Galuzzo 
finds that they cover a common list of aspects of teaching, 
ranging from planning for instruction to presentation to 
classroom management. Their emphasis parallels items in the 
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Lawrence Rudner is the founder 
and president of LMP Associates, 
Chevy Chase, MD, a consulting 
firm concentrating on testing 
issues. Previously a senior 
associate with the Office of 
Educational Research and 
Improvement, Dr. Rudner's areas 
of expertise are test validity, 
measurement theory and computer 
applications. 



professional knowledge section of the NTE. Four states 
include human relations skills and professional behavior and 
three include student evaluation^ items also tested in the 
professional knowledge portion of the NTE. Constitutional 
rights of students and extra-classroom influences on teachers 
and students are tested in the NTE professional knowledge 
section but do not appear in performance assessments. 

Performance assessments are usually conducted by a supervisory 
committee that observes the intern and provides feedback. 
Committee members usually include the building principal, a 
teacher educator, and a supervising classroom teacher* all of 
whom receive extensive training in observing and using the 
performance assessment instrument. The make-up of the com- 
mittee provides a link between teacher preparation programs 
and the needs of the schools. Performance «issessment can 
influence the supervising teachers and teacher educators, as 
well as new teachers. 

If the intern lacks essential skills at the end of the first 
year, there is usually another year to improve. If no im- 
provement is shown, the intern may be terminated after the 
second yean Yet, Galuzzo continues, there is little evi- 
dence that interns who exhibit the specified behaviors will 
cause students to learn more. Observation instruments provide 
a glimpse of behavior in isolation and do not give a complete 
picture of either teachers or students. 

All these efforts arc designed to assure a quality teaching 
force, but their real impact is only beginning to be felt. 
An important paper by Bernard Gifford, University of 
California-Berkeley, entitled "Excellence and Equity" reviews 
both the evidence and the arguments regarding the use of such 
tests with minority teachers. Martha McCarthy's paper on 
"Legal Considerations" also addresses important i'isues about 
the role of tests. The most troublesome issue, however, is 
the extent to which these tests and performance assessments 
adequately reflect the qualities we want teachers to have. 
Ironically, most teats and performance assessments have been 
developed on the basis of armchair reasoning, with little 
evidence that the things they assess in teachers actually 
contribute to pupil learning. 

State initiatives in the assessment of teachers and teaching 
have suddenly become big business. If you would like to learn 
more about issues related to state testing and performance 
assessment programs, contact Lawrence Rudner, LMP Associates, 
3109 Rolling Road, Chevy Chase, MD 20815. Telephone (301)986 
1531 or Eilcne Nicosia, OERI, Room 303P, 555 New Jersey Ave., 
NW, Washington, DC 20208. Telephone (202) 357-6556. 
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UNDERSTANDING 
HOW TEACHERS' 
KNOWLEDGE CHANGES 

by Deborah Ball <ft 
Bill McDiarmid 




Debor&h Ball 



Much of the current debate about teacher education centers on 
what teachers need to know to teach academic subjects, where 
and how teachers can best acquire and develop that knowledge, 
and how teachers* knowledge can be assessed. The debate 
ranges over a variety of assumptions aod perspectives. For 
instance, some educators and policymakers emphasize the 
importance of skills applicable to teaching all subjects and 
argue that these skills are best acquired through careful 
coaching by experienced teachers. Others claim that the best 
preparation for teaching is to stujfy an academic discipline in 
depth and that any well educated person can teach. Proposed 
approaches to assess teacher competence range from checklists 
of skills to tests of subject matter knowledge. 

The existing literature on the requirements of teaching and 
how teachers learn what they need is of little use in these 
debates about teacher knowledge and teacher learning. We do 
not know much about what is taught and learned in teacher 
education programs nor how formal influences on teacher 
knowledge (e.g., university courses) compare to nonformal 
influences (e.g., on-the-job experience). We lack information 
about how teachers who have under^ ken professional prepara- 
tion differ from those who have not. We also lack an under- 
standing about the dimensions of teacher knowledge on which 
teachers must draw and the relati^^'ships among different kinds 
of knowledge for teaching. As a result, arguments for parti- 
cular approaches to teacher education and attempts to measure 
teachers* knowledge are often unconvincing. 

To inform current debates and curriculum reform in teacher 
education, the National Center for Research on Teacher Edu- 
cation is examining what teachers are taught and what they 
learn in 11 diverse preservice, induction, inservicc, and al- 
ternative route teacher education programs. Our work combines 
case studies of programs with longitudinal studies of partici- 
pants* learning. Before, during, and after the program, we 
will survey participants' beliefs, attitudes, and knowledge 
concerning the teaching of academic subjects to diveise 
students. A smaller sample of participants in each program 
will be periodically interviewed and observed as they teach. 

A critical feature of our research design is the fact that we 
collect information on participants over time so that we can 
see whether and how their ideas or practices change and what 
factors sc«tm to play a role in any such changes. Tracking 
these changes requires us to define as clearly and p'-ecisely 
as possible the types of knowledge and behavior that are 
likely to change during teacher education. 
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What Should We ''Follow*'? The lack of consensus among our 11 programs on what teach- 

ers need to know and be able to do complicates our task. For 
instance, some programs emphasize subject matter knowledge 
while others emphasize general skills of teaching. Some 
programs urge teachers to let their pupils take an active role 
while others encourage teachers to structure time, space, ana 
content. 

Given such diversity, how could we develop strategies for 
tracking change that would be broad enough to encompass 
diverse points of view and still be sufficiently focused to 
detect subtle changes over time? Below we describe how we 
have resolved this dilemma. 

Bounding Our Inquiry Focusing on mathematics and writing. Initially, we made two 

decisions that helped to set some boundaries for our work. To 
learn more about the relationship between subject matter and 
pedagogy in teaching and learning to teach, we chose to study 
teacher learning in two specific areas: mathematics and 
writing. Not only do these subjects offer a contrast-they 
are also taught from kindergarten through high school and are 
subjects that pupils often have trouble learning. Since cur* 
rem school practices in math and writing classes frequently 
differ from those recommended by subject matter experts, these 
may also be areas on which some teacher educators concentrate. 



Identifying competing conceptions of nood teaching. Our 
second decision addressed the fact that different, even 
competing, visions of good mathematics teaching and writing 
instruction exist. To ensure that our instruments did not 
favor a particular conception, we needed to aiticulate the 
dominant conceptions of good math teaching and good writing 
instruction. If, for instance, we were to assume that teaching 
writing well meant focusing on mechanics and spelling, our 
instruments would contain primarily questions about teaching 
those conventions. Our chances of detecting changes in 
teachers who focus piincipally on the processes of composition 
would be diminished. Similarly, if we focused our instruments 
on a diagnostic*prescriptive approach to teaching mathematics, 
we would likely miss changes in teachers who were thinking 
about ways to involve students in small-group problem solving. 

For both math and writing, we identified several distinct 
approaches to teaching and identified the knowledge and skills 
each approach required. We also identified the views of 
teaching, learning, ?.nd subject matter inherent in each ap* 
proach. 

What Goes Into Teaching? To understand how teachers learn and change, we had to clari* 

fy what goes into teaching. When teacher educators or policy* 
makers talk about teaching, they generally focus on knowledge 
and skills, often treating them as distinct categories. While 
useful for analytical purposes, this distinction conveys a 
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misleading impression that skills can exist independently of 
knowledge. Abklug good questions and giving clear explan- 
ations, for instance, are often defined as skills, yet they 
draw on teachers* knowledge. Besides acknowledging the 
interdependence of knowledge and skills in teaching, we 
recognize that teaching involves other things as well. For 
example, decisions about when to ask particular types of 
questions depend on teachers' commitments and orientations. 
All these ingredients are reflected in teachers' disposi- 
tions—their tendencies to respond in certain ways under 
particular conditions. 

While teaching is of a piece, (earning to teach occurs un- 
evenly over time. Prospective teachers, for instance, ccmc to 
the university with ideas about what teachers do. While at 
the uiiversity, they acquire knowledge and skills in their 
subject, matter areas. Formal study also shapes their ideas 
about teaching as wr** their commitments and orientations. 
When they begin to teacti, they continue to learn-aboui: 
teaching, pupils, and subject matter. Our interest in 
teacher learning requires that we pay attention to changes 
that occur separately in teachers' knowledge, skills, and 
dispositions as well as changes in how they bring these 
ingredients together in their teaching. 

Knowledge of What? How could we sample relevant knowledge in each of these 

areas? Using our focus on learning to teach mathematics and 
writing, we worked through the following domains of teaching: 
subject matter and curriculum, context, learning and learners, 
teaching and the teacher*s role-seeking to specify essential 
teacher knowledge, skills, and dispositions. In each of these 
domains, we have identified the specific questions most ger- 
mane to our focus on the teaching of academic subject matter 
to diverse learners. 

In thinking about subject ;^atter, for example, we recognize 
that w. need to know what teachers and prospective teachers 
understand about mathematics and writing and how they repre- 
sent their understandings to themselves and to others. From 
any perspective on what constitutes good teaching, teachers 
draw on their personal knowledge cf these subjects. We also 
recognize that we should pay attention to how teachers take 
contextual factors into account, including the community, the 
policies of the school district, the classroom setting, the 
cultural backgrounds of the students, and the time of day or 
year. 

In thinking about what teachers need to know tbout learning 
and learners, we focus on teachers' knowledge ci pupils of 
different ages, developmental levels, and backgrounds, as well 
as their knowledge of their own pupils. We have also identi- 
fied critical questions about teachers' ideas of what it means 
to "learn" something, of how learning occurs, as well as of 
what their responsibilities as teachers are. 

11 
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Strategies fcr Tracking 
Teacher Learning 



Conclusion 



In the area of teaching, we are interested in teachers* ways 
of working with pupils: their repertoire of strategies for 
helping students learn maiht unties and writing that include 
attending to the dNcrsity of learners in the classroom, 
figuring out what pupils know, and deciding what to do in the 
classroom and doing it. In addition, we recognize that we 
should tap teachers' ideas about what they need to learn and 
how they believe they can learn it. 

We have cept in mind that ideas about what teachers need to 
know in any of these areas vary from one perspective on good 
teaching to another. For instance, according to one view, 
the teacher's role is to present clear explanations of mathe- 
matical concepts and procedures; according to another view, 
students must actively construct these under^^tandings for 
themselves. While these clearly differ, both reflect points 
of view about what teachers need to know about learning and 
about their role as teachers. 

To track teacher learning, we developed three instruments: 
a questionnaire, an interview, and a guide for observing in 
classrooms. The questionnaire taps teachers' beliefs and 
knowledge about the dimensions outlined above. For ex'jmple, 
to gel at respondents' ideas about learners, we ask them to 
evaluate a number of mathematics and writing tasks and to 
indicate the tasks they believe are beyond the grasp of most 
six- to eight-year-olds. To elicit the knowledge of mathema- 
tics and writing on which they might draw in teaching, we ask 
subject matter questions embedded in teaching scenarios. One 
such scenario, for instance, asks respondents to evaluate the 
reasonableness of an unusual student response in mathematics. 

The interview is designed to explore our participants' views 
about teaching, to learn what factors they consider when 
performing teaching tasks and their views about helping pupils 
learn mathematics and writing. Wc have developed the inter- 
view questions around tasks of teaching, such as appraising 
students' written work, planning an activity or lesson, and 
responding to a pupil's question. 

Our observation instruments include an observation guide as 
well as pre- and post-observation interview schedules that 
enable us to set what we observe in the context of the 
teachers' overall goals and practice. 

In developing measures of teachers' knowledge and skills, the 
Center has faced the same issues as have others who have 
attempted to develop instruments to assess teacher knowledge 
and skill: what knowledge is relevant, what constitutes good 
teaching, and how to develop strategies to find out what 
teachers know and can do. The way we have addressed these 
issues reflects our dominant purpose as researchers investi- 
gating how teachers learn, rather than as policymakers 
developing strategies for measuring teacher competence or 
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granting teacher licenses. Unlike other assessment efforts, 
we do not need to define standards for teaching performance. 
Instead, our instruments must be sensitive to different con- 
ceptions of good teaching. Moreover, since we are tracking 
changes in teachers over time, we are interested in the shifts 
that occur within particular dimensions-such as a teacher's 
beliefs about students, for instance-as well as in how 
teachers integrate different kinds of knowledge in teaching. 

The data collection strategies we have developed, combined 
with our longitudinal research design, will help us understand 
what people learn from a variety of experiences-prior school 
and on-the-job experience as well as formal teacher education. 
Through questionnaires, interviews, and observations, we will 
be able to record changes over time in teachers* and prospec- 
tive teachers* thinking and teaching. At the same time, we 
will be learning what programs are doing through interviews 
with program personnel, analyses of course and program docu- 
ments, and observations of classes and workshops. Putting 
together data about the programs with information about 
teachers and prospective teachers will enable us to address 
our central research question: What is the relative impact 
of different kinds of teacher education programs on what 
teachers learn? 



Phoco by David Kelp 



Deborah Loewenberg Ball ts a 
senior 'esearcher at the NCRTE. 
Also an elementary mathematics 
teacher, her interests focus on 
the role of subject matter 
knowledge in learning to teach. 

Bill McDiarmid is an associate 
director of the NCRTE and a 
faculty member in the College of 
Education at Michigan State 
University, His interests 
include preparing teachers for 
multicultural classrooms and the 
role of context in teaching. 
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INTERVIEW: SHULMAN ON (Lcc Shulman is a professor at Stanford University. He cur- 
NATIONAL CERTIFICATION rently has a grant from the Carnegie Corporation to develop 

prototype assessments of teachers. Carnegie hopes these pro-" 
by Mary Kennedy totypes can be used by its newly formed National Assessment 

Board to develop methods for assessing teaching competence. 

NCRTE Director Mary Kennedy interviewed him to learn more 

about the assessment system he envisions.) 



How do you envision a national 
certification board working? 
When would teaching be assessed, 
who would do it, and what would 
be the consequences of the 
assessment? 




Ltf Shiilau 



Those are hard questions, Mary, because the board itself has 
just been established, and it's going to be their responsibil- 
ity to make these decisions. Given that we don't yet know 
what the character of its deliberations is going to be, the 
way I answer this question is going to be pretty much pure 
speculation. But it's speculation that we have engaged in 
because, in order to do the work we're doing, we have to 
make some guesses that at least permit us to stipulate "If the 
board does such-and-such, then this is the kind of assessment 
that may follow." 

We've been working under the assumption that teacher educa- 
tion would occur at a university from four to five years, 
maybe six. Following that, at the completion of university 
work most states would grant some sort of provisional license 
that would permit someone to become the teacher of record for 
children; that increasingly you'd find across the states the 
mandate for a beginning year or two of supervised residency, 
toward the end of which you either get a permanent certifi- 
cation or tenure decisions are made-a whole bunch of other 
decisions are made at that point. 

My own current conception of how the board might play out is 
that evidence could be accumulated about the capacities and 
performances of a candidate over time via several sources of 
data that would be accumulated in a portfolio or dossier of 
an individual, only one of which would be performance on 
conventional tests. 

A second would be performance at a performance assessment 
center, where someone engages in teaching-like activities- 
via simuk'^ions, via responding to films or videotapes. The 
virtue of the assessment center is that it is standardized 
like a test, but the response mode is much more like the 
performance that you're trying to predict. 

Th^: third source I see is what we're calling documentation and 
attestation. I'm imagining a situation where, during teacher 
education or during a supervised residency, there would be a 
set of accomplishments that a candidate would have to demon- 
strate. Now, in something like obstetrics residencies, a can- 
didate would have to demonstrate that he or she can perform 
certain kinds of deliveries-a normal birth, a caesarean 
section, a breech birth. These are absolutely essential for 
being a competent obstetrician. So you specify evidence of 
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these as a requirement of residency and obligate the residency 
director to oversee the performance and the satisfactory char- 
acter of the performance before signing off on that particular 
accomplishment. Well now, imagine a board of teaching exam- 
iners establishing a whole series of such teaching benchmarks 
that have to be attested to. It wouldn't matter what the 
sequ r.r^ cS, but sometime during the period of teacher edu- 
cattr 1 an: residency you would have to have someone attest to 
the iaCi ' ;dt this teacher is able to, say, design and manage 
a cooperative small-group teaching activity in his or her sub* 
ject area or whatever else was deemed necessary. This means 
that any workplace that wants to be able to provide the 
opportunity i'or board certification for its new teachers has 
to be prepared to provide these kinds of support services and 
opportunities for teachers. 

And the fourth source 'of data we*re thinking about would be 
direct observations by outside observers of classroom teach- 
ing, which is different than documentation-attestation. What 
I would envision would be that the observers would be 
mentors— the people at the school site— and teacher educators. 
They would not be evaluating teachers' work summatively. They 
would be doing formative evaluation to help you get up to 
standards. Their evaluations would always be helpful eval- 
uations. 

So at least that's the way Vm thinking about it now. 
Carnegie has not only permitted me, but encouraged me to 
think wildly about such things. 



Do you concern yourself at all 
with the possibility that these 
ideas arent feasible? 



Do I worry about it? You better believe I worry about it 
constantly. But we also have to recognize that-to the extent 
that around the country state after state is mandating a 
supervised residency and talking about yoking some kind of 
assessment to the residency-things like this are going to go 
on. So it isn't as if we're creating something out of whole 
cloth. I think you have to boot-strap this sort of thing. 
You have to find good models, you have to publicize the fact, 
you have to train people. And if it works at all, it's going 
to take a generation. But you've got to start somewhere. 
Yeah, it's a hard problem. 



You've given a lot of thought to 
the variety of kinds of hiuwledge 
teachers need, and you've pointed 
out that a great deal of teaching 
depends on normative principles. 
Have you thought about the 
influeuct a national board exam 
could have if it cannot assess 
important aspects of teaching? 



Well, I assume there are parts that are not assessable, 
me make a couple of distinctions. 



Lei 



One is the distinction between the capacity to perform and 
actual performance in a particular situation. At one level, 
you could say that the best possible way of assessing the 
teachers is to observe them teaching in classrooms. And yet, 
teaching in some classrooms is far easier than teaching in 
others. So, in some sense, you have to think separately 



the 



IS 



ERIC 



about assessing the capacity of someone to teach in general 
and observing actual performance in a particular situation. I 
may identify that someone is competent to teach in this 
situation even though Fm not prepared to certify that they 
have the capacity to teach in general. 

Another set of differences: When we ask teachers and teacher 
educators to talk about the characteristics of a good teacher, 
the first cut they give you is a set of character traits and 
dispositions: empathy, love of children, patience. Do we 
have any accurate way of measuring them? For the most part, 
given our current conceptions of assessment, Fd say no. 

A board would have to think about two things. One, are those 
the kfnds of characteristics the absence of which would lead 
you to want to prevent someone from teaching? And, if that 
were the case and if there were oo tests for it, where would 
you place the obligation to screen for those things? I think 
I would place that obligation in the teacher preparation pro- 
gram. And, at the latest, within a residency. I don't think 
the inability to test for something becomes a serious threat 
to the usefulness of the assessment unless you treat what you 
can't test as somehow unimportant. 

So anyway, yes--I don't think that all these are going to be 
equally well assessed in the narrow sense of assessment; they 
won't even all be equally assessable in the broader sense of 
documentation. Loving children-I love some children; I don't 
love others. How are you going to weigh that? Those are very 
hard questions. I really have not tried to grapple with those 
yet. 



What happens to the teacher who 
pays for a teacher education, who 
endures a residency and then 
fails the national board exam? 
This person has already committed 
a lot to this career. 



That would be a very good question. If there were things you 
either must have or you can't teach, then there should be an 
obligation of the teacher education programs or residencies to 
get as reliable an assessment of these as early as possible in 
the process. And about the only one I can imagine that I 
wouldn't feel that way about would be abusive or unethical 
behavior with kids. If, for some reason, there was no evi- 
dence of this earlier on and then late in the residency you 
see someone behaving dishonestly, unethically, or abusively 
with children, and document that-I would feel no misgivings 
at all about the standard being applied. But otherwise, I 
think your point is very well taken. 



YouWe emphasized pedagogical 
content knowledge a lot in your 
writing and in your speaking, and 
Vd like for you to talk a little 
bit about what you mean by that 
term and how that is assessed. 



The notion of pedagogical content knowledge emerged from our 
earlier research. We found that it was generally insufficient 
for teachers to take what they already knew and deliver it to 
students a la Federal Express. They had to learn to represent 
their understanding in some new ways, communicate in new 
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forms, in order to make possible student learning. Much of 
our research has been an attempt to understand what this 
transformation entails, and we call the results of that trans- 
formation-what teachers come to understand about the parti- 
cular topics they teach-pedagogical content knowledge. It 
takes the form of analogies and metaphors and examples and 
demonstrations'-ways of adapting understanding to accommodate 
predictable or observed variations in student characteristics. 

But what*s exciting to us about this notion, despite the 
mouthful that it takes to say it, is that it should distin- 
guish what someone who merely understands the subject 
matter knows from what someone who understands it for 
teaching purposes knows. And that's what it is that distin- 
guishes teachers from non-teachers. 

Now, how would you assess it? Well, I think the fundamental 
way in which you assess it is to create exercises or questions 
or problems that involve the teaching of something in parti- 
cular to someone or some people in particular* You will not 
be able to assess pedagogical content knowledge if you're 
looking for general dispositions. You get it by saying to a 
teacher, "Imagine that you had to teach the following chapter 
on photosynthesis to t group of kids with these characteris- 
tics." Now you're eliciting pedagogical content knowledge, if 
it exists. The assessment does not focus on teaching; it 
focuses on the teaching of something to someone under some 
conditions. 



One of the difficulties of 
metaphors is they carry excess 
meaning that we don*t intend" 
they never are perfect. So to 
some extent, every time a teacher 
uses a metaphor, whether willing' 
ly or not, he or she is also mis- 
leading students a little. Now, 
it*s possible that this could be 
a conscious decision on the part 
of the teacher, depending on the 
teacher's purposes for the les- 
son. It's possible also that it 
would be unintended. The extent 
to which you could assess a 
teacher's use of a particular 
metaphor would depend in part on 
your knowledge of the teacher's 
long-range purpose in the lesson. 
Is thnt something that an observer 
or a documenter can know about? 



Probably not. YouVe probably given an example where, in 
the real world, there is a historical context to every act of 
teaching that can't pc^sibly be captured in any assessment. 
From an assessment perspective, something that subtle would 
probably elude the observer. At a broader level, the unin- 
tended consequences of metaphor also provide an unavoidable 
feature of teaching. But it*s also one of the reasons why you 
try to explain things in multiple ways. You try not only to 
use metaphor and analogy but also examples, demonstrations, 
formal definitions. And that's why, frankly, teaching with 
concrete objects is fairly easy, whereas when you get into 
abstractions, it gets more and more fuzzy and difficult. Ym 
not too much worried about that one for teaching. I think if 
we get to the point where that's a serious question and that 
means that we've solved ill the simpler problems, I'll be very 
happy. 
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Suppose a teacher teaches fifth 
grade for two years during resi- 
dency and then, at the board exam, 
gets a question on teaching first- 
grade students in a very different 
kind of school than this teacher 
had been teaching in. Do you 
think, if a teacher develops a 
repertoire of y^ays of transforming 
subject matter in a particular 
kind of school environment, that 
that repertoire would generalize 
to other environments? 



That's a lovely question because it gets right to the heart of 
what it is that the assessment is supposed to certify someone 
for. If the claim is that the assessment is certifying some- 
one's capacity to teach K through 6, then the fact that 
someone's experience has been limited to fifth-grade kids in 
the suburbs is absolutely no excuse for why they cannot 
demonstrate the capacity to teach third-grade kids in an urban 
center. Now, if I'm evaluating you for the purpose of hiring 
you to teach a fifth-grade classroom or I want to give you 
tenure as a fifth-grade teacher, then that's a very different 
assessment question. But, to the extent that the certifica- 
tion decision is u much broader one, then I would be obligated 
as a candidate to make sure that, whatever my immediate 
experience was, I was competent in the full range of settings 
for which my certificate would empower me to practice. 



Thank you for a very useful and 
interesting conversation. 
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Publications Available 



If you wish to be on the Colloquy mailing list, send your request to the address 
below c/o NCRTE Colloquy List. 

The following publications are now available.* To order, send check, money order, 
or prepaid purchase order to NCRTE Publications, 516 Erickson, Michigan State 
University, East Lansing, MI 48824-1034. Please make check payable to Michigan State 
University. Michigan residents should include 4% sales tax with all orders. Allow four 
to six weeks for delivery. 

Issue Papers 

No. 87-1 Teaching Knowledge: The Lights Teachers Live By. 

M. Buchmann. $3.00. 

No. 87-2 Inexact Sciences: Professional Education and the Development 

of Expertise. M.M. Kennedy. $5.50. 

No. 87-3 Intentions, Problems and Dilemmas: Assessing Teacher Knowledge 

Through a Case Method System. H.L. Barnes. $3.00. 



*If you need multiple copies of any NCRTE publication, you may find it more economical 
to order a single copy and make photocopies of it. Because the NCRTE is federally 
funded, its publications are not copyrighted, and you are therefore welcome to make as 
many photocopies as you need. 



Attention Editors 

You may reprint articles from NCRTE Colloquy in your publication. Just be sure to give 
appropriate credit to the National Center for Research on Teaching and send two copies of 
the issue containing the reprinted material to Sandra Gross, NCRTE Editor, 516 Lrickson, 
Michigan State University, East Lansing, MI 48824-1034. 
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